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Part 1: The Science Behind Communicating 


Introduction 


Imagine it’s your first year on your new college campus. There are so many new 
people to meet, things to learn, and institutional processes to wrap your head 
around. It’s been a tough few weeks, but you are confident that this year will be 
different, that your old habits—procrastination, working alone, not asking 
questions—will stay behind. You’ve even heard that your current institution has 
some cool new technology to help you do better in school. You recall that this 
technology was mentioned at orientation, but you don’t really remember much 
else about it. 


Now it’s halfway through the semester and you just went through your first round 
of midterms. That was tough. Despite having done all the right things, like 
studying in groups, going to office hours, and practicing good time management, 
you still feel like you didn’t do as well on your exams as you would have liked. It 
was pretty hard to give school your all while you were juggling multiple jobs and 
stressed about cutting corners to stay within your budget. And the troubling news 
you got from back home in the middle of midterms didn’t leave you in the best 
headspace to concentrate. "Oh well", you think, you really did do your best, so 
you're probably ok. It’s probably just new student jitters. 


You wake up the day after your last midterm to your phone’s buzzing. There’s a 
message. You don’t recognize the sender, but curiosity gets the most of you, so 
you open it: 


Image 1' 


e000 Sprint > 7:34 AM 100% = 


< Messages (123) 456-7890 _— Details 


Urgent: Due to currently 
failing grades, you are 
not on track to graduate. 
Schedule an 
appointment with your 
counselor immediately to 
remedy the situation. 
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“What. The. Heck. Who is this? How do they know my grades?” A million 
questions race through your head as you scramble to check your grades. Last you 
remember, you were doing ok. Not straight A’s, but you were definitely passing 
everything. Did your midterms really go that badly? 


Shoot. You really are failing. Not all of your classes, but one really important 
course for your major, the one you ve been struggling with the most. This 
message is just the cherry on top ofa series of challenges, self-doubts, and 
obstacles during your education. Let’s face it—you were right when you told 
yourself you weren’t college material. This message is just proof of that. 


This scenario shows the hazards of communicating predictive findings to 
students without care. Though sent with good intentions, this anonymous 
message is counterproductive, increasing this student’s anxiety, self-doubt, and 
even her likelihood of dropping out. 


Predictive analytics has taken higher education by storm, with its promise of 
closing equity gaps, raising student retention rates, and increasing tuition 
revenue by keeping students enrolled. Many colleges and universities have made 
an investment in predictive analytics for student success initiatives, and even 
more are looking into implementing, expanding, and strengthening the 
technology.” 


While having clean data, accurate algorithms, and strong ethical principles are 
vital to putting predictive systems in place to improve student success, these are 
only a start toward creating the institutional change necessary to help more 
students graduate. An algorithm alone cannot create change; action is necessary. 
Getting advisers and other end users to communicate the predictive system 
findings to students is a vital step in successfully using predictive analytics and 
doing so equitably is of utmost importance. 


This report offers research-based guidelines to colleges for engaging in effective, 
ethical, and equitable communication about predictive analytic system findings 
to students. It covers how to approach the first engagement with students: how an 
early alert end user, such as a counselor or adviser, can tell students that a 
problem has been identified, connect them with resources, and create the 
behavior change needed for success. It is also a guide for institutional leadership 
to consider when working with students, faculty, and staff to implement 
predictive analytics at their institution. 
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Why Communication Matters 


Communicating with students can seem easy, intuitive, and low-stakes, and an 
entire report on the subject may seem like overkill. But as the example above 
shows, having good intentions does not guarantee success, and ineffective 
communication can cause serious harm. Extensive behavioral science research 
shows that how institutions communicate to students really matters.’ This is 
especially the case with communicating predictive system findings, as these 
messages relay potentially sensitive information to students. Carefully 
communicating with students matters for the following reasons. 


It matters for student success 


How colleges communicate predictive findings to students can help or hinder 
their success. Communicating effectively can serve to minimize barriers between 
students and resources and help them follow through with their intentions and 
goals. Effective communication can also serve to strengthen students’ sense of 
belonging and confidence in their abilities, an important aspect of student 
success. Poor communication, however, is unlikely to spur change in a student, 
and may, in fact, impede success. For example, unclear communication can leave 
students wondering what they are supposed to do if they fail a course. And as the 
example above shows, a carelessly worded message can make students feel like 
they are not meant for college, and inadvertently push them to drop out. 


Students can also react negatively to messages communicating difficult 
information. Shannon Brady's dissertation for Stanford's Graduate School of 
Education on how students react to probation letters gives us two such examples. 
+ Students are likely to have similar negative reactions to predictive system 
messages that are critical of their academic performance. 


Image 2: Student reactions to probation letters 


G 4 Being on probation sucked...For some time after getting the letter, | felt that | 
didn’t belong. | had already felt that way coming in, but that letter seemed to 
confirm that. For a minute | wanted to drop out.® 


| overreached my preparation by taking [a difficult math class and a difficult 
physics class] in the same quarter. | failed both classes....1 was crushed and 
confused when | received the letter from advising that | had been placed on 
probation....There was a lot of fear, anger and self-loathing....| felt like a 
failure and the vague impersonal language of the letter from some authority 


figure | had never met didn't help me at all.® 9 9 
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It matters for your investment*° 


Predictive analytics can cost a lot of money and social or political capital, so 
appropriately and effectively communicating findings to students is essential to 
ensuring your institution’s investment pays off. Effective, ethical, and equitable 
communications will keep students in school, helping institutions reach their 
retention, graduation, and revenue goals. Careless or ill-prepared messages can 
lead to unmet goals and an inefficient use of the capital invested in predictive 
analytics. 


It matters for educational equity 


How predictive system findings are communicated to students has serious equity 
implications. Today’s students are more diverse than ever: 42 percent are 
students of color and 24 percent have children or other dependents.’ Their needs 
and challenges differ from traditional college students who are younger, tend to 
be wealthier, and can focus exclusively on their academics. Poor communication 
can exacerbate challenges that students face. For example, a message without 
clear instructions on how to make an advising appointment could confuse a first- 
generation student who needs an appointment urgently but does not know how 
to schedule one or have relatives and friends who can help. And a message that 
lacks a growth mindset could exacerbate a student of color’s already strong sense 
of lack of belonging. Communicating predictive findings without care can have 
unintended consequences that add to the challenges today’s students face and 
hurt their chances of graduating. Institutions must use messages about predictive 
system findings as an opportunity to refer students to resources and better 
support students of color, low-income students, and other under-resourced or 
non-traditional students on their path to graduation. 


About Predictive Analytics 


Predictive analytics is the use of mathematical models that use patterns in past 
data to predict or forecast future events. Colleges often use predictive analytics to 
help retain and graduate students. Institutions do this by developing models 
based on the demographic, behavioral, and/or academic data of past students to 
calculate a score for each current student, which represents that student’s 
likelihood of graduating or dropping out. This predictive score, or risk score, can 
change depending on student behaviors, actions, or circumstances. For example, 
if a student fails to meet with an advisor, his or her risk score rises. 


These risk scores are generally not communicated to students. Instead, an end 
user of the predictive analytics system, like an academic adviser, has access to 
what is commonly known as an early alert dashboard, which helps the adviser see 
if a risk score changes significantly. End users typically send messages to students 
urging them to take actions that will lower their risk score and effectively improve 
their chances of graduating. 
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The Science Behind Communicating 


Communicating predictive system findings is especially tricky because it aims to 
inform students of a potentially unpleasant fact while simultaneously persuading 
them to change their behavior. However, using concepts from several subfields 
within behavioral science can help end users and institutions create effective 
messages. 


Behavioral economics and social psychology provide the key concepts and 
techniques that shape these communications. Behavioral economics helps 
explain how individuals make choices, and thus how a message can be structured 


to cause behavior change.’ Social psychology studies how individuals behave in a 
social context, or the way certain factors affect our behavior in relation to others, 
and informs how individuals may interpret a message.” Using behavioral 
economics and social psychology together ensures that messages are effective, 
ethical, and equitable and support students in their educational goals. 


Behavioral Economics 


Behavioral economics is a relatively new field of study that blends economics and 
psychology.’° Contrary to traditional economics, which operates on the premise 
that humans are rational decision makers, behavioral economics argues that they 
are irrational beings that make decisions within their context and available 

11 . oe . “ue 
resources. Concepts such as loss aversion, decision fatigue, and heuristics all 


come from behavioral economics.” 


Behavioral economics helps explain how people’s surroundings lead them to 
make decisions, and more importantly, how to help them make better ones. 
When it comes to communicating predictive findings, behavioral economics can 
inform how a message is structured to help students be successful. What follows 
are several behavioral economics concepts that can make a message more 
effective. 


Social Proof 


People are more likely to do things their peers are doing. For predictive 
analytics purposes, you may be more successful in persuading students to take a 
certain action, like going to the writing center or meeting with an adviser, if you 
let them know other students are doing it too. Sending a message that says “60 
percent of your peers in your major attended a writing center workshop and 
found it helpful” could convince the recipient to do the same. The ongoing 
project, "Nudges to the Finish Line," provides several examples of messages that 
exemplify the use of this concept and others. See Images 3 and 4 for good 
examples of social proofing. 
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Image 3+ 


LC ToLele Mo) g-Lead (ox-) 


@ee00o0 Sprint > 2:13 PM 


< Messages SampleU 


Hi! Did you know 
261,100+ Virginians filed 
FAFSA by this date last 
year? Join your peers 
and visit FAFSA.gov now 
to get the most aid. Can | 
help? 


Source: Nudges to the Finish Line. See note. 


Image 4° 


CToLole Mel e-Lead(ex-) 
@ee00 Sprint > 3:13 PM 


< Messages SampleU 


Hi! Many students used 
resources like the 1st 


Quadrant Math Center 
last year to pass classes 
& move toward 
graduation. Do you have 
the academic support 
you need to succeed? 


Source: Nudges to the Finish Line. See note. 
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Loss Aversion 


Studies of human behavior have shown that people prefer to protect what they 
have over potential gains. This concept is known as loss aversion.*° Take this 
scenario: when given the option of choosing between having (A) a100 percent 
chance of winning $250 or (B) a 25 percent chance of winning $1,000, most 
people choose option A because they would rather be certain of winning a small 
amount than taking the risk of not winning anything.” 


For predictive analytic purposes, sending messages that make students aware of 
what they may lose as a result of inaction is likely to encourage them to change 
their behavior. Telling students that not enrolling in classes on time could cost 
them a spot in required major courses may encourage them to act more quickly, 
for example. However, institutions must use these types of loss aversion 
strategies in an ethical manner. Employing this technique in a dishonest way may 
lead to undesirable consequences for students, such as undue anxiety and a lack 
of sense of belonging.’® Coupling loss aversion with inclusionary and positive 
language can help students be aware of what they may lose while making them 
feel like an important part of the institution. Images 5 and 6 show how to employ 
loss aversion ethically in your messages to students, while Image 7 shows a bad 
example. 


Image 5°” 


CTolele M ol g-Load (ox=) 


e5cooo F 11:27 AM 80% > 


< Messages NewAmericaU _ Details 


Hi! To keep your seat in 
the Spring classes 
you've enrolled in, you 
need to pay tuition by the 
deadline shown in your 
[school account]. Can 
you pay tuition by this 
date? 


Source: Nudges to the Finish Line. 
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Image 6° 


(eC ToLele Mo) g-lend(ox- 


ec0eeo F 01:11 PM 64% > 


< Messages NewAmericaU Details 


Hi [first name], Fall 2019 
registration is open. 
Don’t miss your chance 
to secure a seat in the 
courses you need to 
graduate! Can | help you 
register this week? 


Source: Nudges to the Finish Line. See note. 
Image 7 


s¥-To Mele Lead (ex-) 
ec0ooo F 06:53 PM 45% > 


< Messages NewAmericaU _ Details 


Hi! Tuition must be paid 
by the end of next week. 
Otherwise, you will be 
withdrawn from your 
classes. 
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Minimizing Hassle 


Students have a lot on their plates, so an effective message will make a desired 
change in behavior as easy to accomplish as possible. A predictive analytics 
message should provide clear and simple instructions so that students know 
exactly what they have to do, how to do it, and can act immediately. If you want 
students to set up an appointment with their major adviser, for example, include 
a link to make an appointment or have them reply to the message with times they 
are available and send a calendar invitation. To encourage students to enroll in 
courses on time, include instructions on how to do so in the message. In other 
words, the easier the better. Image 8 is an example of a message that includes 
clear instructions, reduces the mental energy necessary to take action, and 
therefore minimizes hassle. 


Another benefit of designing messages to minimize hassle for students is that it 
may force an institution to simplify institutional processes that are more 
complicated than they need to be. Messages with clear instructions coupled with 
more efficient institutional processes make it easier for students to take action. 


Image 8” 


\CTeofele Mo) g-Lea i (ox-) 


ec0ooo F 12:34 PM 40% &_> 


< Messages NewAmericaU Details 


Hi! Only 2 weeks until 
classes start! Complete 5 
steps to start the 
semester strong & move 
toward your degree. Can 
you commit to 
completing these before 
8/20? 


Source: Nudges to the Finish Line. See note. 
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Timing 

In addition to minimizing hassles, sending messages in a timely manner will 
make them more effective and increase the likelihood that students engage in the 
desired behavior change. Messages sent during high-leverage junctures make it 
more likely that students will take action (see Image 9).” Periods such as course 
registration, midterms, or the FAFSA deadline may be the optimal time to 
contact students. On the other hand, trying to communicate with students during 
a holiday break or summer vacation is unlikely to be effective. 


Image 9” 


Good practice 


e0c0e0o F 11:02 AM 54% > 


< Messages NewAmericaU Details 


Hey, it’s [advisor name]. 
With finals coming up, | 
wanted to check if you’ve 
used SMARThinking, 1st 
Quadrant Math Center, 
or the Writing Center. 
Can | help connect you? 


Source: Nudges to the Finish Line. See note. 


Preserving Choice 


When communicating predictive system findings to students and trying to 
influence their behavior, it is essential to preserve student choice. In other words, 
you do not want students to take a certain action because you left no other 
option. An example of how this can happen is by making something dependent 
on the completion of another task, like enrolling full time in order to receive a 
tuition discount. Some students may not be able to enroll full time, and without 
other options, could choose to not enroll in the institution at all. 
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Another example might be when course enrollment is tied to FAFSA completion. 
While this may seem like a two-for-one deal for the institution (students file the 
FAFSA and enroll in classes on time), it may put students in a challenging 
situation if they are ineligible to file. It could also leave students out of their 
education entirely: if they do not file the FAFSA, they are not able to enroll in 
classes or receive financial aid. Messages that try to change behavior while 
limiting student choice, like in Image 10, are unethical and ineffective. 


Image 10 


Bad practice 
ecoce = 10:44 AM 75% i> 


< Messages NewAmericaU Details 


Hi David, make sure you 
file the FAFSA so you 
can enroll in classes. You 
can’t enroll in classes 
without a completed 
FAFSA, so submit it now! 


Social Psychology 


Social psychology is the study of “the nature and causes of individual behavior in 
social situations” and includes concepts such as the sense of the self, social 
influence, group processes, prejudice, and discrimination.”* It explores how 
humans think and behave in the context of their relationships and greater 
society. 


As it relates to predictive analytics, social psychology helps us understand how 
the content of a message can affect the receivers’ self-conception and their 
relationship to those around them. Using social psychology to create messages 
ensures that communication is both effective and ethical. Implement or be aware 
of the following social psychology concepts when communicating predictive 
findings to students in order to minimize harm. 
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Growth Mindset 


Studies have shown that believing that one’s abilities can change and improve 
with dedicated effort can improve performance, and that students with this kind 
of growth mindset perform better on a task than those with a fixed mindset 
(believing that talent and ability is innate).”” Communicating predictive findings 
with a growth mindset sends a message to students that they have the ability to 
change their behaviors and improve their academic outcomes instead of being 
stuck in their current ways. As a result, students feel more positive about their 
potential despite their academic challenges. 


Image 11 


LC Teofele Mo) e-Lead (ex=) 


eceeo F 10:58AM 84% Mi> 


< Messages NewAmericaU Details 


Hi Michael, exams can 
be challenging, but with 
the right support and 
study skills you can be 
successful. | recommend 
making an appointment 
at the Tutoring Center to 
help you prepare for your 
upcoming midterms: 
www.newamericau/exam 
support 


Image 12 


BAD PRACTICE: Sharing risk rating with students 


Colleges should not disclose risk ratings or predictive scores to stu- 
dents. Although a risk rating shows an end user how likely a student 
is to persist in his or her education based on a series of variables, it 
does not define a student or set his or her success or failure in stone. 
It is simply a forecasting tool to help an adviser help a student. Shar- 
ing a risk rating or predictive score with students will likely make 
them feel bad about themselves and harm their academic perfor- 
mance. It also does not provide them with helpful information they 
can act upon (see Image 11 for a student reaction to finding out she 
was at risk). 
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Image 13 


Rosie Reid v 
@msreidenglish 


| accidentally found out | was an “at-risk” student my 
junior year of high school. | was pretty confused. When 
| asked a counselor why, he named a bunch of factors 
completely beyond my 16-year-old control 


te Rodney Robinson @RodRobinsonRVA - Dec 17, 2019 


There is no such thing as an at rick student. Students are at promise or at potential if 
resources are there. Words matter. twitter.com/VDOE_News/stat... 


9:04 PM - Dec 17, 2019 - Twitter for iPhone 


Source: Twitter 


Interdependent Focus 


Some studies suggest that whether messages are worded to highlight 
independence or communal thinking can affect how students respond. One 
experiment compared admission letters with either an independent or 
interdependent tone (see Image 13 for an independent message followed by an 
interdependent message). An independent tone emphasizes individualism and 
assertiveness, while an interdependent tone emphasizes groups and 
collaborative behavior. Researchers found that letters with an interdependent, or 
more group-oriented, tone led to more low-income student enrollment as 
compared to an independent letter that highlighted individualism.”° The 
interdependent letter highlighted values of community and collaboration that 
aligned with students’ own. Similarly, communication with a greater focus on the 
“we” as opposed to “me” could make it more likely that students change their 
behavior, especially if they come from more interdependent communities, like 
low-income, first-generation, or immigrant backgrounds, because these 
messages remind them that they are part of the school community and will be 
supported.” 
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Table 1: Independent vs. Interdependent Messages 


C C | am delighted that you have decided to attend Stanford University 
and that you think Stanford is the right place for you. For the next few 
years, you will have many opportunities to explore new ideas and learn 
from our superb faculty and from your own personal exploration and 


individual experiences as a student. A tradition of independence: of 
bold students who assert their own ideas, thoughts, and opinions. 


C C | am delighted that you and your family have decided that you should 
attend Stanford University and that Stanford is the right place for you. 


For the next few years, together with the Stanford community, you 
will have many opportunities to explore new areas and to learn from 
experiences and interactions with your peers. A tradition of learning 
through community—bridging academic study with public service. 


Source: Mindset Scholars Network and EdCounsel, “The Role of Student Experience in 
Postsecondary Completion” (Capitol Hill briefing, Washington DC, June 4, 2019). 
Adapted from Stephens, 2018; Stephens et al., 2012; Yeager et al., 2016; and Markus 
ey Kitaya 


Belonging 


Humans have an inherent need to feel that they belong to a group. Lacking a 
sense of belonging has shown to negatively affect peoples’ performance, 
including students.”* A study on probation letters sent to students, for instance, 
found that they were more likely to persist at their institution when letters 
showed care and concern and expressed that students were an important part of 
the institution’s community.’ Students who received a probation letter that only 
described the procedural process of probation without expressing concern were 
more likely to feel like they did not belong at the institution and were less likely to 
persist and earn a degree. 


Communicating concern over behavior or academic performance from 
predictive system findings, like a drop in grades that qualifies someone for 
probation, can cause students to question their place at the institution, if not 
done with care. Messages like that in Image 14 express care for students and their 
outcomes and could reinforce their sense of belonging. 
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Image 14°° 


LC Tofele Mol g-Lea a (ox=) 


ec5ooo F 12:05PM 70% > 


< Messages NewAmericaU Details 


Hi [first_name]! | want to 
help you find a great job 
that will value your 
degree. Have you 
worked w/ Career 
Services to find jobs that 
fit your program & 
interests? 


Source: Nudges to the Finish Line. See note. 


Stereotyping and Stereotype Threat 


In communicating predictive system findings to students, it is important to not 
make assumptions about why someone is struggling in school or what obstacles 
they face based on assumed or known characteristics, such as race, gender, or 
parenting status. Even if shared with good intentions, assuming things about 
students’ experience in college based on stereotypes, as in Image 15, is likely to 
be hurtful to students, damage your relationship with them, and could imperil 
their academic progress. 
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Image 15 


|S F-Yo Mo) g-Yoad(ox-) 


ec0ooo F 12:05PM 70% > 


< Messages NewAmericaU _ Details 


Hi Madison! We know 
that as a Pell Grant 
recipient you had fewer 
resources growing up. 
You may be struggling as 
a result, so follow this 
link for some study tips! 


Profiling and stereotyping students can lead to other unfortunate outcomes 
beyond damaging a sense of belonging, such as tracking students into certain 
majors or professions that pay less. When acting upon a predictive finding, an 
end user may feel inclined to help students succeed by tracking them into 
“easier” majors, like public health instead of pre-med. While there is nothing 
wrong with switching majors, it is problematic to advise an “easier” route without 
first providing support for students in their chosen major, especially if the student 
is underrepresented in that field. 


Stereotyping can also lead to stereotype threat, the social phenomenon where 
individuals are or believe they are at risk of conforming to a stereotype about 
their social group.” Stereotype threat can hinder an individual’s performance on 
a task. In various studies, individuals who took an exam and held a marginalized 
social identity (such as a woman taking a science exam with two men, or one 
person of color taking an exam with two white people) actually performed worse 
on the task because they were stressed about confirming a stereotype.” 


End users should be careful to avoid communicating bias and stereotypes to 
students when sending messages about predictive system findings, whether 
unintentionally or not. If students receive a message saying that they are unlikely 
to graduate because they are low-income or a student of color, this message 
could trigger a series of thoughts, behaviors, and events that leads to 
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underperforming. Bias training and thorough research and testing of messages 
can help prevent this. 


Self-Fulfilling Prophecy 


Stereotyping can lead to a self-fulfilling prophecy, the social psychology 
phenomenon where an individual thinks something will happen and then acts in 
a way that makes that outcome occur.* Self-fulfilling prophecies can be self- 
imposed or imposed by others. In the context of higher education, students who 
learn they have a low predictive score may take that to mean they are not “college 
material” and, asa result, study less, perform poorly, and eventually drop out. 
Although a low predictive score does not explicitly state that a student is not 
college material, the student can come to understand that and then engage in 
counterproductive behaviors that make failure seem inevitable. Poorly 
communicated predictive findings can lead to the very behaviors you are trying to 
change or prevent. 
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Part 2: Creating the Message 


Behind the Scenes 


Now that you have a better understanding of the science behind effective 
communication and what can cause failed messages, it is important to learn how 
to put that into practice. This section will cover the process for creating successful 
messages and highlight aspects that make them effective. Tailor these guidelines 
to fit your own institutional culture, resources, and needs. 


1. Create Your Team 


A diverse team will help create and deploy communications about predictive 
system findings most effectively. Ideally, teams should be as diverse as possible 
considering available resources. Social diversity (such as race/ethnicity, gender, 
etc.) and diversity of professional background (like communications experts and 
those in advising or institutional research), are both important for the creation of 
effective messages. Consider hiring undergraduates to help provide a sense of 
how messages will be heard by students and bring more diversity to your team. 
Diversity will bring important nuance to how you construct your messages. 


2. Select the Messenger 


Who the message comes from matters. The best messengers are entities and 
individuals with whom your students have a relationship or rapport. The stronger 
the students’ relationship with the sender, the more likely they will respond to the 
message and engage in the desired behavior change. 


It is important that messengers identify themselves. Otherwise, students may 
ignore the message or think it is spam. When messengers do identify themselves, 
but have little rapport with students, the results vary. Some experiments have 
been successful in changing student behavior with messages that come from an 
entity with weaker ties to students, like an advising office or The Common 
Application (Common App).** Other experiments used messengers that students 
were more familiar with, such as individual counselors.*° My analysis of the 
literature on nudging in higher education suggests that students are more likely 
to respond to and follow through with messages that come from senders they 
know and trust. For example, Georgia State University creates this connection by 
introducing students to their advisers on the first day of class and characterizing 
the chatbot as the school's mascot, Pounce. The university has found that if 
students have a face to go with the name and a relationship to the messenger, 
they are more likely to feel an affinity and use them as a resource. Below is an 
example of how to introduce a messenger to a student. 
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Image 16°° 
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< Messages NewAmericaU Details 


Hi [first_name], I’m 
[advisor name], a 
[institution name] 
advisor, & I’m here to 
support you to finish your 
degree! Is there anything 
| can help you with now? 


Image 17°”3* 


IN THE FIELD: Messenger matters. 


In one study, the Common App sent nudges, or short messages 
aimed to change behavior, to students to remind them to submit 
their FAFSA application, and only a small positive effect was found.*” 
In a different study where nudges came from counselors, nudges had 
a much stronger effect on student FAFSA completion, likely because 
the students receiving these messages had a relationship with the 
messenger.*® 


These two studies suggest that having a messenger with rapport is key to an effective nudge. If 
students do not know or trust the messenger, nudges can easily be ignored, changing little. 


3. Choose the Modality 


The platform, or modality, on which you deliver your messages also matters. 
Messages can be sent via email, text, campus app, social media, campus portals 
or print. Select whichever students are most likely to read, trust, and respond to. 
This decision may or may not require an institutional shift to a new platform, but 
in the end the choice should ensure students receive your message in a timely 
manner through a channel that is not overcrowded with messages from different 
parts of the college. 


While text messaging can be more effective than email, the best platform is the 
one that is best for your institution and students. Each is “best” depending on 
what it is being compared to, so do some research on what platforms work at your 
institution. Sometimes leveraging existing platforms by focusing on critical 
points in the student timeline can make them more efficient and effective. 
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Image 1 


939-4041 


Trendy Communication Tools in Higher Ed 


Various forms of communication have garnered the attention of the higher education field: 
nudges, chatbots, and campus apps. Each plays a different role and has different costs and 
benefits. 


NUDGES 


A nudge is a strategy aimed at changing people’s behavior without 
limiting their choices or changing their incentives.*? These one-way 
communications sent from a messenger to individuals tend to be easy 
and cheap.* In higher education, this can be a text reminding 
students to file the FAFSA or meet with a counselor. This strategy 
came to fame with Richard Thaler and Cass Sunstein’s 2008 book 
Nudge, and now it seems that most communications for student 
success in higher education are characterized as nudges. “ 


If done correctly, nudges can push people toward changing their behavior in a way that is 
beneficial to them, like participating more in online course discussion boards. They are relatively 
affordable to use and can have significant, immediate results. However, it is hard to measure 
longer-term behavior change as a result of a nudge. In higher education, for instance, it may 
never be known if a one-time nudge to study more results in a long-term change in study habits. 


CHATBOTS 


Chatbots are artificial intelligence platforms that enable written 
conversation between an individual and a computer. They can “exist” 
through text messaging or on a website. Chatbots are generally 
staffed by humans as well, and in higher education, staff such as 
advisors can be part of the conversation when the technology can't 
answer a question. Examples of some education technology 
companies that provide chatbot services are AdmitHub and 
SignalVine. 


Chatbots, which can be very accessible for students if delivered on a texting platform, can be 
beneficial in many ways. They can free up counselors by answering simple or logistical questions 
about institutional processes, help students access information easily and at their convenience, 
and gather data on what students’ needs and experiences are. At the same time, chatbots pose 
logistical and ethical challenges. If not introduced properly, for example, students may be unclear 
on whether they are communicating with a computer or a real person. 


CAMPUS APPS 


Campus apps enable users to access institutional information on one 
platform. Campus apps should be interactive and mobile-optimized, 
instead of just a copy of the institution’s website. Platforms such as 
Canvas, institutional email, chatbots, and more can be integrated 
into a campus app and offer a centralized, customizable platform for 
students. At the same time, implementing and using campus apps 
can be difficult. Centralizing information and communication can be 
challenging at institutions if their structure and culture have 
historically been decentralized, as is common. 


And staff and students could be resistant to this institution-wide change. Getting students to use 
a campus app successfully is also challenging because it requires several steps, and each 
presents a behavioral barrier. Students could download the app, but procrastinate on registering 
or logging in. Each of these steps might require guidance or persistent messaging to ensure it 


gets done. 
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4. Create and Test Your Message 


After choosing a platform, your team has to create the messages you will send to 
students or select which vendor-created messages you will use. 


If you are creating your own messages, testing these on students will help clarify 
what works. Use resources you already have to test messages and gather 
feedback. Student employees or focus groups work well and are efficient, but 
students should be compensated for their participation. Another option is to 
engage in A/B testing—testing and comparing two options—to create successful 
messages. Finally, work across departments to gain important insight into the 
impact of messages. Check-ins and informal feedback from staff across 
departments are especially important in decentralized institutions, as the impact 
of a message on one group of students could be different on another. The 
creation and testing of messages is an iterative process, and you should expect to 
return to this step several times. 


Thorough analysis of the student body throughout the communication life cycle, 
including demographics, behaviors, culture, and needs, is important for crafting 
a successful message about predictive findings. Insufficient research and 
understanding are likely to lead to an unsuccessful message. Without proper 
analysis, you can send messages that are irrelevant, unhelpful, or even offensive 
to students out of ignorance of student likes, wants, and needs. 


Image ig? he 


IN THE FIELD: The importance of understanding your population 


The College Board’s Realizing Your College Potential study aimed to 
increase low-income student enrollment in selective colleges 
through nudges and other interventions. “? In this experiment, stu- 
dents received easily digestible information about selective colleges 
they could apply to and encouragement for doing so. These messag- 
es were delivered through email or text. Some students received ad- 
ditional messages to help them overcome psychological barriers to 
applying to these colleges. 


But in spite of the use of various modalities and tailored messages, students did not significantly 
change their enrollment patterns. A key reason why the experiment was unsuccessful was be- 
cause researchers made faulty assumptions about how low-income students select the college 
they want to attend. 


Researchers operated from the common assumption that financial barriers and a lack of 
knowledge about college prevented students from applying to and enrolling in selective, compet- 
itive colleges. “* However, many low-income students intentionally choose to attend less compet- 
itive institutions because that may allow them to save money or enroll in college while still at- 
tending to their family responsibilities.“ A more holistic understanding of how low-income stu- 
dents select the institutions they apply to could have led to more targeted efforts and communi- 
cations that better served the population.” 
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Image 20°° 


IN THE FIELD: Getting teenagers to eat healthier foods 


Getting teenagers to eat healthier foods is challenging, but one re- 
search study found that tapping into teens’ need for rebellion was ef- 
fective in getting them to eat healthier. “© Teens who participated in a 
workshop that taught them that junk food companies bombard them 
with enticing ads to get them to buy their products, and that eating 
healthily ruins these plans, were more likely to make healthy food 
choices at the cafeteria compared to those who were told only that 
eating healthy is good for long-term health. Knowing your audience 
may expose unique ways to create behavior change. 


It is likely that your communications about predictive system findings will have to change over 
time because what students know, need, and learn will change. Being nimble and flexible in your 
approach to communicating predictive system findings will help make these changes easier and 
getting positive results more likely. 


5. Decide When to Press “Send” 


Your team should brainstorm about when the best times are to reach out to 
students. Consider sending messages during high-leverage points, like course 
registration, to get students to complete an action. Avoid sending messages very 
early or late at night, on weekends or on holidays. Using data on how students 
respond to your messages can also help determine the most favorable times to 
send a particular message. You are likely to find that sending text messages 
during holidays tends to result in higher unsubscribe rates or unread messages. 
Finally, institutions should consider developing and implementing a campus- 
wide communication strategy so that important messages, such as those from a 
counselor, are not lost in the noise. 


6. Train Your End User 


Before and during outreach to students, ensure end users, such as counselors, 
faculty advisers, and administrators, are trained at interpreting and 
communicating predictive analytics findings. Users should be able to understand 
the institution’s strategy with predictive analytics, what goes into the model, 
what predictive scores mean, and how to communicate findings effectively and 
ethically. Work on the front end will help ensure successful communication of the 
tool’s findings. 


The Message 


Once all of the research and preparation work has been done, implement the 
following components to make a message successful. 
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Message Structure and Content 


START WITH A PROPER GREETING 


A proper greeting uses the student’s name and identifies who the messenger is. 
Personalizing a message helps increase response rates and show that the 
institution cares about the student.*” Identifying the messenger helps establish 
and strengthen the student-messenger relationship. 


Image 21*° 


Good practice 


ecooe = 11:24PM 


< Messages NewAmericaU 


Hi [first_name], I’m 
[advisor name], a 
[institution name] 
advisor, & I’m here to 
support you to finish your 
degree! Is there anything 
| can help you with now? 


Source: Nudges to the Finish Line. See note. 


Be sure not to leave the student’s name blank. Careless messages hurt rapport. In 
Image 22, an institution did not fill in the blank for a student’s name and the 
student did not appreciate that. 
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Image 22 


Flo Mo) ¢-Lead (ox: 


Sarah 

@skg_18 
nothing makes me think “wow, my college really cares 
about my mental health!” like calling me “[first_name]” 
:) <3 


Text Message 
Today f 


Hey [firstname]! We know finals 


are a stressful time on campus. Did 
you know the Wellness Center has 
walk-in counseling available during 
business hours? 

Stop by Wellness to check out 
Stress Free Zone or speak with a 
counselor. All are welcome! https:// 
nyQED (stress-free: 
zone 


11:21 AM - Dec 13, 2019 - Twitter for iPhone 


Source: Twitter 


MAKE THE MESSAGE INTERACTIVE 


Making the message interactive will make it more likely that students take the 
desired action. You should include a link to the action you want the student to 
take, such as a link to make an advising appointment. Including an interactive 
component reduces the amount of actual and psychological barriers to task 
completion for students and allows them to complete the task immediately. If 
you cannot make a message interactive, include clear and concise instructions on 
how to complete the action so that students know exactly what to do instead of 
leaving them wondering how to fix the problem. 
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Image 23” 
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< Messages NewAmericaU _ Details 


Hi [first_name]. If you 
have federal student 
loans, take 2 minutes to 
answer 5 Yes/No 
questions and find the 
repayment plan that 
works best for you: [link] 


BE BRIEF 


Lengthy emails are likely to be overlooked or ignored. It is best to be brief and get 
to the point so that students can take action. Students, like all people, do not have 
the time to decipher lengthy emails. 


BE POSITIVE BUT REALISTIC 


Messages should be positive so that students are not discouraged, especially 
because communicating predictive system findings can often be sensitive in 
nature. Making students feel like they are not the only ones struggling, 
encouraging them to get help, and emphasizing that they have the capacity to 
improve their academic performance will minimize harm and increase the 
likelihood that they take action. 


At the same time, it is important to be realistic, especially when students are 
struggling. Being overly positive can diminish the gravity of a problem, blind 
students to the real consequences of their situation, or set unrealistic 
expectations. Be sure to balance a positive tone with a realistic approach so that 
students are both encouraged to continue their education but are not set up with 
false hopes for their success. 


Case in point: at one institution using predictive analytics, faculty advocated fora 
small but important distinction in how they communicated to students. Students 
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who were performing poorly in classes were originally told that “it’s not too late” 
to improve their grades. However, due to several factors, including faculty course 
policy, it was more accurate to tell students that “it may not be too late” to 
improve their grades. This change was successfully implemented to reflect a 
message that was both positive but realistic. 


Similarly, do not tell students something that is not true. Ultimately, predictive 
analytics is intended to help students and the university succeed. This can make 
communicating less than ideal findings to students challenging for end users. 
While it may be tempting for end users to manipulate findings to sound more 
positive, they must be honest with students. Both students and institutions will 
benefit from the truth. 


GIVE THE VALUE PROPOSITION 


It can be challenging to encourage behaviors that do not have an immediate 
payoff. Including an explanation about why an action is important in a message 
could help students understand why it benefits them and convince them to do it. 
For example, making an appointment for tutoring can seem unnecessary; 
students may convince themselves that they will just study more next time. But if 
the message makes the value of meeting with a tutor clear and explicit by using 
behavioral economics strategies, students will be more likely to take the desired 
action. As always, it is important to be positive in phrasing the value proposition 
of an action so as not to provoke anxiety. 


Image 24°° 
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< Messages NewAmericaU _ Details 


Hi [first_name]! Only 2 
weeks until final exams. 
A study plan can help 
you do well on finals & 
graduate on time. Can 
you set aside 2 hrs each 
day to study? 


Source: Nudges to the Finish Line. See note. 
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AVOID-ISMS 


It should go without saying: avoid “-isms” and discrimination. Comments that 
are racist, sexist, able-ist, homophobic, etc. are unethical, ineffective, hurtful to 


students, and will likely result in other negative consequences for the institution. 


Be especially careful not to include microaggressions, or comments that are 
subtly prejudiced toward marginalized groups, like in Image 25. Ensure both 
message creators and end users are up to date on their bias training to avoid 
microaggressive or outright problematic messages. Having a diverse team can 
also help recognize and flag when a message might be offensive. 


Image 25 
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< Messages NewAmericaU _ Details 


Hi Chloe! We know that 
women tend to 
underperform in STEM 
majors. Sign up for a 
session at the Peer 
Tutoring Center to get 
help and catch up! 
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Conclusion 


Predictive analytics for student success has the potential to make positive change 
in higher education and students’ lives, but it comes with the responsibility of 
ensuring that findings are appropriately communicated from end users to 
students. This seemingly simple task of communication requires institutions to 
be well informed so that they can take advantage of the full potential of their 
investment in predictive analytics to help students graduate. 


At the same time, predictive analytics should be seen as a tool in a toolbox, not a 
solution to higher education challenges on its own. Predictive analytics should be 
implemented alongside broader institutional and policy reforms that help make 
the higher education system more equitable and easier to navigate for students. 
Otherwise, predictive analytics will continue to be a Band-Aid over a larger 
problem. 


The field of predictive analytics and communicating its findings in higher 
education is rapidly evolving and many institutions do not have the resources or 
infrastructure to hire a vendor or engage in rigorous message testing. Sharing 
what your team has found to be effective for communicating predictive analytics 
findings will help other institutions help their students, regardless of institutional 
access to resources. Embracing a collegial nature will help more students and 
institutions be successful. 


As the country and its higher education system experience rapidly evolving 
changes and become even further stratified, retaining students and helping them 
finish their degrees is more important than ever. Predictive analytics and 
effective, ethical, and equitable messages can be a part of institutions’ strategies 
to fulfill their mission of educating students and closing equity gaps. 


This research-based guide can help institutions better communicate with their 
students. By putting in work on the front end and being intentional, institutions 
can minimize harm toward students and maximize their opportunity to obtain a 
degree. 
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Appendix: List of Interviews Conducted 


Bryan Cook, PhD, Vice President of Data and Policy Analysis, APLU 
August 7, 2019 

Andrew Magliozzi, CEO, AdmitHub 

August 27, 2019 and December 12, 2019 

Amelia Parnell, PhD, Vice President for Research and Policy, NASPA 
September 18, 2019 

Jeffrey Meece, EdD, Vice President of Strategy, Mongoose 

October 25, 2019 

Jamie Riepe, Chief Revenue Officer, SignalVine 

October 29, 2019 

Marty Major, Director of Partner Success, Gecko Engage 

October 28, 2019 

Eric Stoller, Vice President of Digital Strategy, Gecko Engage 
October 28, 2019 


Caroline Altman-Smith, Deputy Director of Education, The Kresge 
Foundation 


November 6, 2019 

William Moses, Managing Director of Education, The Kresge Foundation 
November 6, 2019 

Jeff Mutimer, Vice President of Marketing, SignalVine 

November 21, 2019 

Jeannine Sato, Director of Communications, Duke University 

November 25, 2019 

Chris Hopkins, Chief Strategy Officer, Ready Education 


November 26, 2019 
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Benjamin Castleman, PhD, Associate Professor, University of Virginia 
and Director, Nudge4Solutions Lab 


December 11, 2019 


Tracy Gale, Communications Manager of Information Technology, 
University of Florida 


December 12, 2019 


Tiffany Mfume, PhD, Assistant Vice President for Student Success and 
Retention, Morgan State University 


December 16, 2019 


Shannon Brady, PhD, Assistant Professor of Psychology, Wake Forest 
University 


December 20, 2019 


Lindsay Page, PhD Associate Professor of Research Methodology, 
University of Pittsburgh 


December 20, 2019 
Lynn Brabender, Assistant Director of Office of Urban Initiatives, APLU 
January 30, 2020 


Samantha Levine, Learning Collaborative Manager of Office of Urban 
Initiatives, APLU 


January 30, 2020 


Elizabeth Adams, PhD, Associate Vice President of Undergraduate 
Studies, California State University, Northridge 


February 12, 2020 

Lynn Brabender, Assistant Director of Office of Urban Initiatives, APLU 
March 2, 2020 

Jamie Schnur, Director of Higher Education Support Services, Hobsons 
March 3, 2020 

Catherine See, Starfish Account Manager, Hobsons 


March 3, 2020 
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Timothy Renick, PhD, Senior Vice President for Student Success, Georgia 
State University 


March 6, 2020 


Carol Cohen, Assistant Vice President for University Advisement, 
Georgia State University 


March 20, 2020 
Lindsey Fifield, Chatbot Project Director, Georgia State University 


March 20, 2020 
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